Search results for "Validity assessment"

Statement validity assessment: Inter-rater reliability of criteria-based content analysis in the mock-crime paradigm

2005

Methods. Three raters were trained in CBCA. Subsequently, they analysed transcripts of 102 statements referring to a simulated theft of money. Some of the statements were based on experience and some were confabulated. The raters used 4-point scales, respectively, to judge the degree to which 18 of the 19 CBCA criteria were fulfilled in each statement. Results. The analysis of rater judgment distributions revealed that, with judgments of individual raters varying only slightly across transcripts, the weighted kappa coefficient, the product-moment correlation, and the intra-class correlation were inadequate indices of reliability. The Finn-coefficient and percentage agreement, which were cal…